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I. Introduction 


In the past, management scientists have mainly focused their attention 
on the design of decision systems aimed at the solution of programmable 
and recurring problems. Such areas as inventory control and refinery 
scheduling can now be managed almost automatically by computer driven 
mathematical models. Although such models may require extensive as 
well as complicated mathematical manipulations, yet in their normal 
use they are rather simple in that the fundamental relationships en- 
compassed by these models are well prescribed. Martin Starr (Starr 1966) 
refers to problem situations which can be depicted by such deterministic 
planning and control models as "fully-constrained,"^ because their 
associated environments, although these may be evolving, they are con- 
sidered to be perfectly predictable and all sequences of events are 
known with certainty. 

Next in terms of complication come planning models which are 

probabilistic in nature either in their inputs (both data and assumptions), 

or in the fundamental mathematical relationships among the variables in- 

2 

corporated into these models. These planning models which one may 
classify as partially constrained (Starr 1966), lead to tentative con- 
sequences requiring the value judgment of the decision maker before a 



8, terminology for classification of planning models. 


As the reader may have already observed, we are classifying these models 
on the basis of the structure chosen by the decision maker in his effort to 
choose a course of action and not on the basis of how these decision situa- 
tions could have been structured. Obviously fully constrained models could 
be set up as probabilistic models. With the exception of pointing this out 
we shall not delve into the question of the factors affecting choice of 
models, nor in the evaluation of the degree of comprehensiveness of such. 



- 2 - 

choice is made. Finally, in extreme cases, planning problems may be de- 
picted by threshold-constrained systems (Starr, 8) in which the sequence 
of events is speculative, the environment must be forecasted, and some 
potential outcomes may be catastrophic. 

The problem used in our experiments was of the "partially constrained" 
type. We wanted to find out how executives could deal with capital invest- 
ment and competitive pricing decisions under conditions of uncertainty. 

3 

Many writers have pointed out the value of formal planning and also 
stressed the necessity of using structured situations as a stepping stone 
to higher level (unstructured) planning. Of course this is easier said 
than done, and the average manager partly because of necessity but mainly 
for escape finds himself spending proportionately much more time on 
operational than on planning problems. This relative aversion toward 
planning is due both to psychological as well as methodological reasons. 

No doubt planning enforces self discipline, requires persistent effort, 
provides standards which can be potentially used by superiors for control 
and accountability, exposes errors and as such decreases privacy, enforces 
integration and cooperation across organizational functions and activities, 
with all their human-behavior consequences; and finally demands a resolution 
of the inherent conflict between the specific plan as a secure basis for 
implementing action and the plan as a temporary mechanism for measuring 
deviations, learning from experience and then updating the underlying 
planning model. ^ 

_____ 

Among others, see Ackoff, R. L., 1, Ansoff, H. I., 2, Starr, M. K., 8, 
and Zannetos, Z. S., 10. 

Sjhat we are saying here is that "security" and shielding by the plan 
is only temporary and the manager cannot survive with either chaos or 
complete regimentation. 
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Zannetos (10 p. 13) has identified three reasons why planning 
problems are more difficult than operational problems. They are: 

1. Absence of structure 

2. Absence of many factual elements 

3. Presence of uncontrolled environmental conditions 

Realizing the complexity of planning problems and the difficulties 
associated with designing useful computerized decision systems to aid 
the decision maker in solving such problems, we embarked upon a research 
project which called for the design, implementation and experimental 
use of a computerized planning model. The risk analysis technique (Hertz, 4) 
was chosen as the basis of the model because it incorporated (1) a facility 
for structuring problems, (2) a methodology for utilizing subjective esti- 
mates of elements for which no factual information exists and (3) a tech- 

> 

nique for introducing risk and uncertainty through the use of subjective 
probability distributions. While the risk analysis technique does not 
eliminate the three major reasons why planning problems are difficult, it 
does recognize them and operates on them, and thus mitigates this onerous 
managerial task. 

It was decided from the outset to implement the risk analysis model 
on an interactive graphical display computer terminal. This decision was 
made for the following reasons: 

1) Interaction was necessary since the system operates on 
subjective inputs provided by the user. 

2) Since planning decisions are partly based on value judgments 
and subjective inputs, the user must have the capability to 
ask "what if" type of questions and receive answers in real 
time. In this respect we distinguish between two classes of 
situations. If the "what if" question involves changing the 
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values of some inputs but does not involve changing basic 
underlying assumptions, it is defined as sensitivity testing. 

If it involves changing most, or all, of the values of the 
inputs and also changing the basic underlying assumptions, 
then it is defined as a new alternative. These two uses of 
the system will be discussed in more detail later. 

3) Since the risk analysis techniques utilizes probability distri- 
butions as inputs and outputs, it was felt that graphical 
representation of the distributions would be more meaningful 
and useful than would tabular or parametric representation. 

The end product of the systems development effort was a software package 
called the Interactive Risk Analysis Model (IGRAM) System which is available 
on the Compatible Time Shared System (CTSS) at MIT's Computation Center. 5 

After the IGRAM System was developed during the early part of 1970, 
two controlled experiments were conducted which attempted to measure the 
impact the IGRAM System had on a decision-making process. The next portion 
of this paper is devoted to a discussion of those experiments. 


II. The Experiments 
1) Introduction 

A review of the available literature on planning reveals that it 
mostly centers around the need for planning models and to a much lesser 
extent on the implementation of models. There is, however, a dirth of 
literature on the observed impact these models have had on the decision 
making process. In an effort to fill this deficiency in the literature, 

5 For a detailed disciussion of the design and implementation of the IGRAM 
System, the reader is referred to Beville, J., et.al., 3. 
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it was decided that controlled experiments should be conducted using the 
IGRAM System. Our objective was to gather data from experiments which 
could be used in analyzing the impact of the system on decision making 
(both the results and the process) and also in gaining insights for the 
design of future man-machine systems. 

2) The Experimental Design 

TWo sets of subjects were available (business executives), so 
the experiment was replicated twice. The experimental design was com- 
pletely randomized with two treatments, the latter being: (a) solving 

a complex pricing problem using manual solution techniques and (b) solving 
the same problem with the aid of the IGRAM System. In each replication 
of the experiment the subjects were assigned to two-man teams at random. 
Members of the odd numbered teams solved the problem with manual methods 
first and then solved it with the aid of the computerized IGRAM System. 
Members of the even numbered teams solved the problem with the aid of 
the IGRAM System first and then solved it with manual methods. This 
ordering scheme was intended to neutralize the learning effect of 
having to solve the problem twice. Figure 1 illustrates the sequence 
of steps taken by the teams during the conduct of the experiment. Copies 
of the referenced questionnaires are contained in Appendix A. 
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Figure 1 


Experimental Procedure 
Order of Events 


All 

Odd Numbered Teams Teams Even Numbered Teams 

Answer Questionnaire 

#1. 

Receive Copy of Case 
Receive Hertz* Article 

Solve Case Manually 
Answer Questionnaire #2 


Receive Copies of IGRAM 
Users' Manual 

Briefing on and Demonstration 
of IGRAM System. 


Solve problem with IGRAM 
System 


Answer Questionnaire #3 


Answer Questionnaire #4 
Solve Case Manually 
Answer Questionnaire #5 


Class Discussion of 
Problem and IGRAM 
System 


* 


See Hertz, D. 


B., 


(4). 
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III. The Subjects 

a) Senior Executives 

The participants in the first replication of the experiment 
were twenty businessmen holding high level positions within their 
organizations and who had been attending the Spring 1970 session of 
the Senior Executive Program, at the Sloan School of Management at 
M.I.T.. The program is nine weeks long and is designed to update the 
senior executives' knowledge of modern management techniques and expose 
the participants to present research and future trends in the field of 

management. The executives were in their seventh week of the nine week 

program when the experiment started. During the program, the executives 

were housed in Endicott House, a suburban mansion belonging to M.I.T.. 
With this arrangement, the participants had unlimited opportunities 
for exchanging ideas. On the whole, the Senior Executives were in- 
terested in the experiment and enthusiastic about their participation, 
which incidentally was voluntary. 

b) The Greater Boston Executives 

The participants in the second replication of the experiment 
were twenty-five businessmen from the Greater Boston area who had been 
selected by their firms to participate in the Spring 1970 session of the 
Greater Boston Executive Program. The program is conducted by the Sloan 
School of Management and lasts fifteen weeks. The participants attend 
classes only during each Friday of the fifteen weeks, and perform their 
regular duties at their firms during the other four days of the work 
week. The four class periods each Friday are designed to update the 
executives' knowledge in the areas of economics, managerial planning, 
information and controls, labor economics and social responsibility. 
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This program is not as extensive in its coverage as the Senior Executive Program 
Furthermore, these executives generally hold positions of lesser responsi- 
bility than do the Senior Executives, are not as mature managerially, are 
younger in age and comparatively less well educated than the Senior 
Executives. The Greater Boston Executives were in their sixth week of 
the program when the experiment started. 

Unlike the Senior Executives (SE), the Greater Boston Executives (GBE) 
lived at home during the program. Practically the only contact they had 
with each other was during the Friday classes and during activities 
associated with the experiment. 


IV. Expected Impacts 

If we use for reference the phases of the decision making process as 

L ’ 

postulated by Simon — intelligence, design, and choice--the IGRAM System 
was expected to affect only "design" and "choice," the intelligence, i.e., 
problem definition, having been taken care of by the material distributed 
to the subjects. In particular we were hoping that the system would help 
the user in the "design" phase by: 

(1) Facilitating the structuring of the alternative courses 
of action 

(2) Bringing to bear on the problem the decision-maker's subjec- 
tive estimates of the values of uncertain variables. 

(3) Performing the complex computations required to accomplish 
(2) above. 

(4) Facilitating sensitivity testing. 

^See Simon, H. A., 7. Other writers on the - subject (especially Ackoff 1, 
Starr 8, and Zannetos 10) tend to look at these phases as aspects of the 
planning process. 



As for "choice" activity, the system was not so much intended to 
make the choice, but hopefully aid the decision-maker in evaluating more 
effectively the alternatives generated by: 

(1) Providing more information than do manual methods. 

The system accomplishes this mainly by providing a graph 
of the probabilistic distribution of the net present 
values rather than just the expected value of such which 
is normally provided by manual methods. 

(2) Displaying information in a format which is easy to under- 
stand. The system for example provides graphs rather than 
tables of values or mathematical descriptions of the various 
probability curves. 

(3) Allowing the decision-maker to comprehend the impact which 
his subjective estimates of risk and uncertainty have on 
computed expected outcomes. 

(4) Making it easy for the subjects to change the values of the 
planning and decision variables and thus easily evaluate 
and choose among alternatives. 

o 

V. Hypotheses to be Tested 

The hypotheses which we wanted to test were mainly as follows: 

1) The subjects will tend to examine more alternative courses of 
action when using the computer than they will when solving the case manually. 

^Morton's work (Morton, M.S.S. 5) tends to support this hypothesis which 
seems to be rather widely accepted. 
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2) The subjects will tend to have more confidence in their computer- 
aided decision than in their manual decision. This proposition appears 
& priori to be reasonable for the following reasons: 

(a) The system allows the subjects to bring to bear all 
of their knowledge--objective, subjective and risk 
estimates--on the evaluation of the future consequences 
of each alternative course of action. Thus the results 
will tend to appear to them as more dependable and 
"scientific." 

(b) The system gives the subjects better knowledge of the 
structure of the problem, and through sensitivity testing, 
an idea of the relative impact exerted by the key variables 
on expected outcomes. They will feel therefore that they 
have a firmer grasp of the problem and that the computer- 
aided decision rests on a firmer foundation. 

(c) The system carries out the calculations and gives the 
decision-maker additional quantitative information concerning 
the degree of risk and uncertainty associated with a proposed 
course of action. The additional information in this case 
consists of the graphs of the various terminal probability 
distributions, which is normally not available under manual 
methods. 

(d) The system displays probabilistic information in formats, 
namely graphs, which are easier to understand than alternative 
formats, such as tables or listings of distribution parameters 
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(e) Psychologically the subjects may feel insecure and tend 
to accept the results of manipulations they do not very 
well understand. Furthermore, the feeling that systems 
are designed by "experts" who undoubtedly store in the 
computer the best decision-making models known to them 
tends to encourage too much faith in the results of such 
models . 

3) The decisions made with the aid of the computer, that is, the 
courses of action recommended, will differ from these suggested by manual 
methods. This is felt to be true because: 

(a) The system provides the subjects with more information 
concerning risk and uncertainty than they will be able to 

g 

generate by using manual methods. 

(b) When using the computerized system, the subjects will tend 
to rank the risk and uncertainty inherent in an alternative 
higher as a factor influencing their decision than they will 
when using manual methods. This appears logical since 
measures of risk even in their simplest form, e.g., variance, 
and general shape of the distribution of net present values, 
will be nearly impossible for the subjects to generate by 
using manual methods. They, therefore, will not tend to 
appreciate the significance of risk in decision making. If 
this hypothesis is proven then certain aspects of the 
educational value of the computerized system are proven also. 

g 

The underlying assumption here is that the executives are able to comprehend 
and process this information. Schroder, H. M. , et.al., 6, have found that 
beyond a certain point of environmental complexity people tend to process less 
information. In our case the structure provided by the system reduces the 
complexity so we do not' believe that we will be reaching beyond the maximum 
information processing point. 
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VI. Measurement Instruments 

Three types of instruments were used to collect data during the 
experiment. They were: 

1) A set of five questionnaires. 

2) Written solutions to the case which were handed in by the teams, 
one for manual solution and one for the computer-aided solution. 

3) An informal debriefing of each set of subjects in order to 
obtain their reactions to the IGRAM System in particular and to 

. the use of planning models in general. In addition, the research 
assistants who observed the subjects during their computer terminal 
sessiops gathered valuable informal data. 


VII. The Problem 

The problem which each subject had to solve twice arid which was 
implemented on the IGRAM System was a capital investment case study. TVo 
basic alternative courses of action were open -- 1) discontinue producing 
a textile product which was a part of the firm's full line of textiles, 
sell off the associated equipment and inventory, and collect the associated 
accounts receivable; or 2) continue producing the product for the next 
four years (the equipment would be worn out at that time) and, given that 
decision, further decide upon the best pricing strategy to follow. 

The problem was implemented on the IGRAM System in the form of a 
problem tree, as shown in Figure 2. The "Stay in Business" alternatives 
1 and 2 represent two different pricing strategies over the four year time 
horizon. Such elements as Industry Volume and Market Share would be adjusted 


in each case to correspond with the particular pricing strategy chosen, 


instance, a high pricing strategy might call for a smaller industry volume 
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and a smaller market share than would a low pricing strategy. All inputs 

to the elements on the end points of the tree are in the form of cumulative 

9 

subjective probability distributions. 

VIII. Analysis of the Experimental Data . 

We will now analyze some of the data which were collected on the 
experiments and compare the odd numbered teams to the even numbered teams 
within each replication of the experiment. In addition, comparisons will 
be made between the results generated by each of the replications. 

As the reader may recall, the primary difference between the odd 
numbered and even numbered teams, within each replication, was that the 
odd numbered teams solved the problem manually before solving it with the 
aid of the IGRAM System, while the even numbered teams first solved it 
with the aid of the computer system. The main difference between the two 
replications of the experiment was the difference in the managerial experience 
and education of the two sets of subjects. As we have already stated, 
in general the Senior Executives (SE) had more management and general 
education than did the Greater Boston Executives (GBE) . With these 
differences in mind, let us now examine the experimental data. 


1. The Decision Made 

The case that we used for the experiments was structured so that the 
net expected values of the alternatives were very close together. This was 
done in order to elicit the value judgments of the subjects, their utility 

o 

For a more complete description of the IGRAM System, the reader is 
referred to Beville, J., et.al., 3. 
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regarding risk and uncertainty and any other intangible elements which 
might creep into their decision-making process. Given the above experi- 
mental bias, the results of Table 1, which represent the number of changes 
in decisions after the experiments, tend to substantiate our hypothesis that 
decision are affected by the computer. A little further we will look into 
the reasons behind most of these switches, but right now let us look at the 
data of Table 1. In the context of the question asked, the decisions of 
a subject were defined as "switched" if one recommended that the company 
discontinue making the product, while the other decision recommended con- 
tinuation of operations. 

Of the sixteen Senior Executives who used the system and completed 
the experiments, only two reported a switch in their decision. 10 In sharp 
contrast nine out of twenty one Greater Boston Executives switched. This 
may lead us to hypothesize that the Senior Executives are possibly more 
conservative than the relatively younger Greater Boston Executives, or that 

l 

the latter were not as mature and thorough in their original analysis and 
therefore learned relatively more after the first solution than did the 
Senior Executives. In our estimation, it is mainly the relative immaturity 
of the Greater Boston Executives in making high-level decisions which is 
manifested in the results and this is supported by evidence presented in 
Tables 3 and 4. 

If we were to assume that all the subjects belong to the same universe 
then we would expect to find approximately five Senior Executives switching 
instead of two and six Greater Boston Executives instead of nine. Also, if 
the 'treatments" were neutral we would expect to find about five switches 

ioT 

There were twenty Senior Executives in the program but only sixteen volunteered 
for the experiment. The respective figures for the Greater Boston Executive 
p-rogram were 25 .participants and 21 volunteers. 

11 The figures are rounded to the nearest integer. 
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Table 1 


Differences Between Computer-Aided 
and Manual Decisions 


Senior Executives 


Odd Teams Even Teams Total 


Switches 


2 

0 

2 

No change 


7 

7 

14 


Total 

9 

7 

16 


Greater Boston Executives 




Switches 

5 

4 

9 

No change 

4 

8 

12 

Total 

9 

12 

21 

Grand Total 

18 

19 

37 


Odd Teams solved the case first manually and then used the 
system while the Even Teams did the opposite. 


Note: 
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occurring among the odd teams and six among the even teams instead of 
seven versus four respectively. 

While we do not wish to strain the limited amount of data we have 
for statistical significance, it is evident to us that the data suggest 
that there exist differences between the Senior Executives and the Greater 
Boston Executives and also between the odd and the even teams. 

In Table 2 we present the qualitative decision made by those who 
switched. The data reveal that the six of the ten subjects, for whom 
complete information exists, chose to continue the production of the 
product when they used the system and chose to discontinue its production 
when they solved the problem with manual methods. Five of these six 
subjects belonged to the odd-numbered teams which solved the case manually 
at first and then used the computerized system. 

In order to ascertain the reasons behind the switches we asked the 
subjects to tell us which decision they preferred and to qualify the change 
of decision. Out of eleven, ten provided information and indicated that i 

all ten would stick to their computer-aided decision. The reasons behind / 

the "switches" are listed in Table 3. 

At first glance there appears to be a discrepancy between some of the 
evidence contained in Tables 2, 3 and the "absolute" faith in the computerized 
decision shown by those who switched. Especially in view of the four 
switches among the even-numbered team members who, as the reader may remember, 
solved the probelm manually before using the computer. What the subjects are 
actually saying is that the data they developed manually dictated a decision 
opposite to the one they arrived at with the aid of the system. However, if 
they had to choose between the two different solutions, they would stick to 
their computer-aided decision. , 
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Table 2 

Differences Between Manual and Computer-aided 
Decisions for those subjects who 
Changed their Decisions 

Number Reporting Decisions 


Senior Executives 




Odd 

Numbered Team Members 


Manual 

Computer-aided 



2 

Discontinue 

Continue 

Greater 

Boston Executives 


Manual 

Computer-aided 

Odd 

Numbered Team Members 

4 

Discontinue 

Continue 



1 

Continue 

Discontinue 

Even 

Numbered Team Members 


Computer-aided 

Manual 



1 

Discontinue 

Continue 



2 

Continue 

Discontinue 



1 

No data 

available 
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Table 3 


Reasons for Switching Decis: 


Reasons Provided 
Senior Executives 


The subject made errors in 
manual solution 

The assumptions were not exactly 
the same 


Greater Boston Executives 


The subject did not consider the 
same costs for every input re- 
quired in both solutions 

The computer provided more data 

The subject made errors in manual 
solution 

No reason given 


2 


1 
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The results of Table 3 further indicate that the computer system forced 
two of the odd numbered team members within the Greater Boston Executives 
to use only the relevant variable costs in the analysis of the alternatives. 
These subjects had used some fixed costs in their previous manual solutions, 
but subsequently the computer guided them to the relevant differential 
costs. In other words the computerized system imposed a methodological 
discipline. Three even- numbered team members did, however, indicate that 
their computer-aided decisions were based on more data than were their 
manual decisions. In this respect we must assume that the additional data 
were the graphs of the distributions of net present values since that was 
the only new information which was generated by the system. Unless of 
course the availability of easy computational power was mislabeled as 
"more data." 

We must admit that we expected more dramatic differences between the 
two modes of decision making. Possibly, the limited range of differences 
between the manual and computer-aided decisions can be explained in part 
by the abstraction of realism in the experiment. A decision maker in 
real-life situation would no doubt possess a great deal more intuition 
and judgment than did the experimental subjects, and he might therefore be 
able to specify better probability distributions than those devised by the 
subjects. Also, as already explained, the expected values of the two major 
alternatives posed in the problem were intended to be so close together that 
any decision would have to be made on the basis of the differences in the 
shapes of the net present value distributions, "intangibles" and qualitative 
assumptions. When proper manual methods were used, the expected net present 
values for the two major alternatives — continue or discontinue--were almost 
identical. However, when the subjects used the system they tended to make 
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optimistic sales projections which tipped the scales in favor of continuing 

production of the item. This fact made it easy for the subjects to make 

their computer-aided decision without having to differentiate between the 

alternatives on the basis of the shape of their corresponding distributions 

of net present values. Both the transcript of the subjects' use of the 

computerized systems as well as the data of Tables tends to substantiate 

this explanation. As can be seen in Table 2 of the seven odd-numbered team 
12 

members who switched, only one decided to discontinue with the other six 
deciding to stay with the product. 

2. The Decision Making Process 

(a) The hypotheses that subjects would rank risk higher as an influence 
on their computer-aided decision than on their manual decision received 
mixed support. The Senior Executives responses supported the hypothesis 
while the Greater Boston Executives responses did not support it. In addition 
to a possibility that this hypothesis is not valid there may be two other 
explanations of the reaction of the Greater Boston Executives: (i) Being 

less mature, as managers these subjects may not have appreciated the 
importance of risk in decision making and (ii) the question attempted to 
make a ranking distinction between risk, short-term, and long-term profit- 
ability as criteria for managerial choice. It Is quite probable that these 
subjects were unable to make such fine distinction especially since risk is 
not completely independent from short and long term profitability. 

12 

The - reader is reminded that the odd-numbered teams attacked the problem 
manually at first and then worked with the system. 
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Table 4 



Number of 

Alternative Courses of Action 

Examined 



SE 





Computer-a ided 


Manual 

Both 


Odd 

Even 

Odd 

Even 

Odd 

Even 

m=5.56 1 

m=3.43 1 

m=5.00 2 

m=2.86 2 

m=5 . 34^ 

m=3 . 14^ 

v=2.25 

v=0.858 

v=6.29 3 4 

v=0.8 3 

v=4.00 5 

v=0.508 5 

n = 9 

n = 7 

n = 8 

n = 7 

n = 17 

n = 14 


GBE 





Computer-aided 


Manua 1 

Both 


Odd 

Even 

Odd 

Even 

Odd 

Even 

m=3 .67^ 

m=2.66^ 

m=3.70 8 

m=2.50 8 

m=3.68 10 

9 cq1° 

m= 2.58 

v=6.00 7 

v=0.427 7 

9 

v=5 . 34 

v=0.455 9 

- 0/11 
v=5.34 

v=0.417 11 

n = 9 

n = 12 

n = 9 

n = 12 

n = 18 

n = 24 


Where in=mean of the sample j v=variance of the sample; and n=sample size 


1. T-test on difference means is significant at 0.1% level. 

2. T-test on difference between means is significant at 5% level. 

3. F-test on ratio of variances is significant at 2.5% level. 

4. T-test on the differences between the means is significant at the 0.1%, level. 

5. F-test on the ratio of the variances is significant at the 1% level. 

6+8. Difference is significant at the 10% level. 

7+9+11. Ratio of variances is significant at the 1%, level. 

10. Difference is significant at the 5% 0 level. 
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(b) The results in both replications only weakly supported the I 

hypothesis that decision makers would examine more alternative courses 
of action when using the system than when using manual methods. Table 4 
contains the results of the experiments. 

As can be seen in Table 4, the only case where a set of teams re- 
ported examining fewer alternatives when using the computer system than 
when using manual methods (3.67 versus 3.70) was the odd-numbered GBE teams. 
All other teams reported an increase although in no case the mean 
differences were statistically significant. The startling result revealed 
by these data is that the odd-numbered team members in every instance, \ 

examined significantly more alternatives than did the even-numbered team j 
members. In the case of the Senior Executives, the odd-numbered teams 
examined 5.56 computer-aided alternatives versus 5.00 manual alternatives 
while the even-numbered teams examined 3.43 and 2.86 alternatives respectively 
The overall average number of alternatives examined was 5.34 for the odd- 
numbered teams versus 3.14 for the even-numbered. These data are 
statistically significant at the 0.1 °L level. Similar results are revealed 
by the Greater Boston Executives experiments although not as strongly. 

Another startling observation is the difference in the variances between the 
odd and the even-numbered teams, which differences are again statistically 

significant at the 0.17, level. The fact that the odd-numbered teams solved • 

\ 

the problem manually first, seems to have had an important influence upon , 

i 

their decision on how many alternatives to examine. 

A possible explanation of the results of Table 4, is that the method 
initially used by a decision-maker sets the style he will use in subsequent 
decision making including the general number of alternatives he will examine 
before arriving at a decision. When the odd-numbered teams first approached 



24 


the problem manually, they were possibly not very confident in their manual 
methods of considering risk and uncertainty. Therefore, they examined 
many different alternative courses of action before they made their final 
decisions. The even-numbered teams, however, first solved the case with 
the aid of the computerized system. They possibly had a high degree of 
confidence in the way the computer system handled risk, so they did not feel 
the need to examine many alternatives. The high level of confidence of the 
even numbered teams, therefore, may have led them to a strange sense of 
security which resulted in their examining fewer alternatives, on average, 
than did the odd-numbered teams. We presented the results to the participants 
without expressing our views as to the cause and they provided the same 
explanation. 

Hence, the method first used by the decision-maker sets the style he 
will use in subsequent decision-making. Given their respective styles , as 
reflected by the number of alternatives they considered, three of the four 
groups did, however, examine more alternatives when they used the system 
than when they used manual methods. We must stress again, however, that for 
each group , the mean differences between the number of alternatives examined 
(computer-aided versus manual) were not statistically significant, contrary 
to widely held notions. 

.(c) In accordance with out previously stated hypothesis, the subjects 
in both replications reported that they had more confidence in their computer- 
aided decision than they had in their manual decision. This was reported 

13 

in spite of the fact that only 11 of 37 subjects changed their decisions. 

__ 

As we have already stressed, the subjects may have changed decisions or 
not simply on the basis of the output data of the method used. However, even 
those who switched decisions after they attacked the case manually still had 
more confidence in the computer-aided decision. 
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So, even in the cases where the manual and the computer-aided decisions were 
the same, (in fact even in the case where the subsequent manual method 
dictated a switch), the subjects had more confidence in their computer- 
aided decision. The causes for the higher degree of confidence in the 
computer-aided decisions were traced to: 

(1) The subjects confidence in the "risk analysis method" of 
handling uncertainty. 

(2) The additional information the system provided in the form 
of graphs of the distributions of net present values. 

(3) The structure imposed on the problem by the system. 

(4) The ease with which the subjects understood the graphical 
information. 

(5) The subjects ability to perform sensitivity tests, that is, 
ask "what if" questions of the system. 

(6) The "fact that the system was designed by experts who must have 
for sure incorporated in it the latest management science 
techniques." 

The results under (b) and (c) above may also indicate another phenomenon 

.14 

which Schroder has observed in his experiments . He found that after a 
certain degree of complexity in the environment the subjects tend to process 
less information and yet feel more confident in their decision. So it could 
be that trying fewer alternatives, as did the even-numbered teams, was a 
manifestation of an inability to cope with the complexity of the computerized 
case which resulted in a bias toward a lower level of information processing. 
These are among some challenging hypotheses which we intend to explore in 
the future. 

14 

Related to one of the authors. 



26 


IX. Implications of Planning Models for Management 

Based on the results of the experimental use of the IGRAM System, 
we feel that planning models, such as the IGRAM System, hold an exciting 
promise for management in the future. The following are some of the uses 
which we foresee: 

1) As a central tool . The IGRAM System requires the user to input 
his subjective estimates of future states of nature, and decisions are made 
based on the information generated from those estimates. All of this 
information can be stored in machine readable form and can be quickly 
accessed by the user. It seems natural that managers will want to review 
the progress of projects which were approved with the aid of the IGRAM System 
or one similar to it. These reviews could be made very easily by examining 
the original assumptions and estimates and comparing them with later 
appraisals of the situation. Current data could be periodically introduced 
into the system to facilitate such reviews. Quick, convenient, real-time, 
computer-aided reviews would help the decision-maker detect problems — 
assumptions which are not coming true or critical estimates which are off 
the mark--and use the Monte Carlo simulation technique to project the con- 
sequences of these problems and evaluate alternative solutions to the problem. 

As the managers use the system to help control the project, they would also 
learn more and more about the nature of the variables which are key to the 
project, and eventually develop a diagnostic system rather than do post mor terns . 

2) As an educational tool . A system, such as IGRAM, could be used 
to teach new managers the procedures used in making decisions in the past-- 
the elements considered the assumptions and estimate made — and the process 
through which they should progress in making future decisions. The use of 



the system in this manner will undoubtedly reveal the weaknesses of past 
decisions as well as their strength. The use of such a system appears to be 
a much more efficient method of training managers than any method in use today. 

3) To improve consistency in decision making . The structure and 
discipline imposed on problem solving should produce more consistent and 
efficient decision making. Within a firm, it would be possible to have 

the specialists in making certain types of decisions develop their own models, 
which models in turn can be used by other managers within the firm whenever 
they are faced with one of those types of decisions. Another advantage 
such a system would provide is that of constituting a permanent memory. 

Experience would not be lost when a "good" manager leaves the firm. His 
decision-making technique would be left behind for others to study and learn. 

Also the "weeding out" of obsolete methods, through automatic updating of planning 
models, would eliminate a lot of the inefficiencies we find in practice today. 

4) To improve communications . With a system such as I GRAM, qualified 
people can specify the structure of the problem and then experts from the 
various relevant areas can enter their estimates of the variables related 

to their area of expertise. Methods could even be devised to reconcile the 
differences between the "expert opinions" of several experts on a single 
variable in those cases where there is disagreement. 

5) A tool for consensus decision-making . If we go a step beyond the 
resolution of differences between expert opinions we can see how such a system 
could facilitate the resolution of differences between recommended courses 

of action in cases where there are many decision makers. Since all inputs 
to the system must be explicit, any differences of opinion may be traced 
to individual inputs, to the system, or to individually held values. In 
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either case, the use of the system can help to isolate the cause of the 
disagreement and hopefully aid in its resolution. We see interactive 
systems with global models as means of increasing communication and under- 
standing of complex interrelationships. 

Obviously, the surface of the field of planning model development 
and use has only been scratched. Much is yet to be discovered and validated. 
The need is for more detailed, documented experimental work using prototype 
models. The sooner we discover the secrets of planning models, the sooner 
managers will be able to harvest the fruits of their use. 
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